AD-A057  915  SRI  INTERNATIONAL  MENLO  PARK  CA  NAVAL  WARFARE  RESEAR— ETC  F/G  9/2 

AUTOMATED  DATA  SYSTEMS  (ADS)  MANAGEMENT  METHODOLOGY.  VOL.  II.  A— ETC(U) 
DEC  77  D L HARVEY#  T M KEEN.  E H MEANS  N00014-76-C-1119 


END 


OATI 

FILMED 


m 


10 


1 


I 


NATIONAL  BUREAU  OF  STANDARDS 
ttcuocorr  resolution  test  chart 


!)DC  FILE  CQPL  AD  A 0 5 7 9 1 5 


1 


PH 


FTAT"  ***  "’rr-j 
u *»•  V+  •' « * U 

* tv,;-  ‘; 

l^jn  >'■  (}  . ” 

a ta’ 

Lyi  I?  r • 


AUTOMATED  DATA 
SYSTEMS  (ADS) 
MANAGEMENT 
METHODOLOGY 


’ 


Vol.  II:  Automated  Data  Systems 

Project  Evaluation  Methodology 


Naval  Warfare  Research  Center 


Final  Report 


December  1977 


D DC 

(n^Pr^'PPH  nrp 

AUG  7 1978 


I 


R 

i ; 

!l! 

! i i 


By:  David  L.  Harvey,  Terrance  M.  Keen,  Edward  H.  Means, 
William  Schubert,  and  Graham  F.  Wallace 


Prepared  for: 

Commandant  of  the  Marine  Corps 
Headquarters  Marine  Corps 
Washington,  D.C.  20380 

Attention:  Chairman,  “Automated  Data  Systems  (ADS)  Management 
Methodology"  Study  Advisory  Committee 


Contract  N0001 4-76-C-1 1 1 9 


Reproduction  in  whole  or  in  part  is  permitted  for  any  purpose  of  the 
United  States  Government. 


SRI  International 

333  Ravenswood  Avenue 

Menlo  Park,  California  94025 

(415)  326-6200 

Cable:  STANRES,  Menlo  Park 

TWX:  910-373-1246 


Dli:  V ' 1 • Vi  'V  A 

Apr  . . pu-  iio  Ui-  aaej 
. Unlimited 


78  08  04.  049 


ITOMATED  DATA  SYSTEMS 
BS)  MANAGEMENT 
iTR  OTOLOGY  , 


Vol.  IL^Automated  Data  Systems 
^ project  Evaluation  Methodology* 


Naval  Warfare  Research  Center 


«■ 


Byipavid L/Harvey, Terrance M^een, Edward H.ldeans,  \ 
/WilliamBchubert^^rtDGrahamFjWallace  ' \ 


Prepared  for: 

Commandant  of  the  Marine  Corps 
Headquarters  Marine  Corps 
Washington,  D.C.  20380 

Attention:  Chairman,  “Automated  Data  Systems  (ADS)  Management 
^Methodology"  Study  Advisory  Committee 


Contractual  4-76-C-1 119  / 

' 

SRI  Project  5765-4 

Reproduction  In  whole  or  in  part  is  permitted  for  any  purpose  of  the 

United  States  Government. 

q-xo  0 8 

04  049 

From: 

To: 


DEPARTMENT  OF  THE  NAVY 
HEADQUARTERS  UNITED  STATES  MARINE  CORPS 
WASHINGTON.  D.C.  20380 


IN  NCftY  NCFER  TO 

CCIS-20-cls 
5230/1 
2 4 JUL  1978 


Commandant  of  the  Marine  Corps 
Distribution  List 


Subj : Automated  Data  Systems  (ADS)  Management  Methodology 
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Enel:  (1)  ADS  Management  Methodology  Study 

1.  The  Automated  Data  Systems  (ADS)  Management  Methodology 
Study  was  initiated  to  produce  a set  of  procedures  and 
documentation  to  be  utilized  in  the  Conceptual  Phase  of 
ADS  development. 


2.  The  enclosed  study  has  accomplished  the  stated  objective. 
The  methodologies  proposed  by  the  study  would  be  beneficial 
to  any  ADS  development  effort.  Certain  portions  of  the  study, 
such  as  the  Resource  Estimating  Procedure,  will  require 
changes  to  reflect  recent  Department  of  Defense  instructions. 


3.  The  Marine  Corps  intends  to  utilize  the  ADS  Management 
Methodology  Study  as  a base  for  revision  of  current  orders 
pertaining  to  ADS  development.  Appropriate  modifications 
will  be  made  to  Marine  Corps  orders  to  reflect  changes  in 
policies  and  procedures. 

4.  A copy  of  this  letter  will  be  affixed  inside  the  front 
cover  of  each  of  the  subject  study  prior  to  its  distribution. 
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PREFACE 


Analysis  by  the  Marine  Corps  has  indicated  that  the  growing  demand 
for  ADS  has  outstripped  the  available  management  capability  to  support 
application  of  the  existing  technology.  For  example,  the  growing  demands 
for  systems  documentation  and  analysis  during  ADS  development  may  be 
beyond  the  current  capabilities  of  the  Marine  Corps  to  support  satisfac- 
torily, because  of  the  shortage  of  qualified  personnel  and  inadequate 
staffing  at  the  Headquarters  level.  One  manifestation  is  that  the  cost 
and  time  necessary  to  develop  and  implement  an  ADS  are  consistently  under- 
estimated. 


It  appeared  that  the  most  immediate  remedy  might  result  from  first 
tackling  the  problem  of  ADS  development  management.  Current  management 
is  guided  by  the  Marine  Corps  Automated  Data  Systems  Manual  (ADSM) . 
However,  specific  statements  of  duties,  tasks,  operational  definitions, 
and  deliverables  for  individuals  and  work  teams  are  not  specified  in  the 
ADSM  in  sufficient  detail  to  allow  for  a high  degree  of  efficiency  for 
the  average  developer. 


As  a consequence,  HQMC  contracted  with  SRI  International  to  conduct 
a systems  analysis  study  of  the  conventional  management  concepts  expressed 
in  the  ADSM.  The  study  was  to  develop  a methodology,  operational  defini- 
tions,* and  a definitive  statement  of  tasks  and  duties  for  ADS  development 
Benefits  accruing  to  the  Marine  Corps  from  a successful  completion  of  the 
study  effort  were  to  be  both  economic  and  operational: 


1.  Economic  — if  the  efficiency  of  automated  systems  development 
is  increased,  there  will  be  a reduction  in  the  time  and  cost 
of  developing  any  given  ADS. 


"Operational  Definition:  An  operational  definition  is  a specification 
of  the  activities  of  a worker  or  a team  in  developing  and  using  a com- 
ponent of  an  ADS  development  plan.  Alternatively,  an  operational 
definition  assigns  meaning  to  a concept  by  specifying  the  activities 
and  operations  necessary  to  produce  a usable  component  of  an  ADS 
development  plan.  Thus,  an  operational  definition  provides  a bridge 
between  theory  and  application. 
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2.  Operational — if  the  effectiveness  of  automated  systems  develop- 
ment is  improved,  the  systems  that  are  developed  in  the  future 
will  better  conform  to  valid  mission  requirements. 

The  specific  objectives  of  the  research  were  to  accomplish  the 
following : 

1.  Operationally  define  the  management  and  analysis  tasks  and 
duties  that  must  be  performed  in  the  preparation  of  an  ADS 
Development  Plan. 

2.  Operationally  define  a procedural  methodology  for  developing 
ADS  objectives  and  associated  measures  of  objective  fulfill- 
ment . 

3.  Develop  specific  methodologies  needed  to  guide  developers  and 
decision  makers  for  ADS  systems  so  that  the  anticipated  value 
of  the  proposed  system  can  be  assured  and  its  total  cost  impact 
estimated  and  controlled. 

4.  Assist  HQMC  to  translate  the  study  results  into  procedures 
and  policies. 

The  study  was  begun  in  mid-1976  and  completed  in  late  1977.  It  con- 
sisted of  the  following  five  tasks: 

Task  1:  Objective-Writing  Methodology 

Task  2:  Resource  Requirements  Estimating  Methodology 

Task  3:  ADS  Action-Document  Preparation  Procedures 

Task  4:  ADS  Project  Evaluation  Methodology 

Task  5:  Integration  and  Refinement  of  ADS  Management  Methodology. 

The  results  of  Tasks  1,  2,  and  3 were  reported  on  in  individual  Technical 
Notes  issued  during  the  course  of  the  project,  namely,  NWRC-TN-73, 
"Objective-Writing  Methodology  for  USMC  ADS  Development,"  July  1977; 
NWRC-TN-72,  "Resource  Requirements  Estimating  Methodology,"  May  1977; 
NWRC-TN-75,  "Concept  Phase  Procedures  and  Action  Documents,"  September 
1977. 

This,  the  project's  final  report,  has  been  published  in  two  volumes. 
Volume  I addresses  Study  Tasks  1,  2,  and  3 and  is  designed  for  use  by 
originators/users  in  presenting  the  case  for  a proposed  ADS.  The  material 
originally  contained  in  the  Technical  Notes  has  been  restructured, 
simplified,  and  oriented  as  much  as  possible  to  the  nontechnical  ADS  user. 
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Those  desiring  more  detailed  explanations  of  the  results  of  Tasks  1 
through  3 are  encouraged  to  consult  the  Technical  Notes. 

^This  document , "Volume  II  of  the  final  report^synthesizes  and  corn- 
ea 

piles  the  results  obtained  unde^\  Study  ^asks.^J. , 2,  and  3 in  order  to 
address  the  objectives  of  Tasks  4 and  5.  It  *is  suggested  that  an 
originator/user  of  a proposed  ADS  concept  can  profit  from  reading  both 
volumes  of  the  report  since  the  evaluation  criteria  presented  in  Volume 
II  should,  if  possible,  be  satisfied  by  the  proposal  documents  treated 
in  Volume  I. 

The  study  was  conducted  within  SRI  International's  Naval  Warfare 
Research  Center,  A.  Bien,  Director.  The  Project  Leader  was  David  L. 
Harvey.  Study  team  members  included  Terrance  M.  Keen,  Edward  H.  Means, 
William  Schubert,  and  Graham  F.  Wallace. 


^1 


TABLE  OF  CONTENTS 


PREFACE iii 

ILLUSTRATIONS  ix 

TABLES ix 

SECTION  1.  INTRODUCTION  1 

SECTION  2.  MARINE  CORPS  OBJECTIVES  AND  ADS  3 

SECTION  3.  DECISION-MAKING  CONCEPT  PHASE  ACTIVITIES  5 

SECTION  4.  PLANNING  THE  DECISION  PROCESS  11 

SECTION  5.  THE  RS  DECISION 17 

SECTION  6.  THE  FS  DECISION 19 

SECTION  7.  THE  ADS  DP  DECISION 21 

7.1  General 21 

7.2  Formation  of  the  ADSDP  Evaluation  Team  ....  21 

7.3  Analysis  of  the  ADSDP  on  its  Own  Merits.  ...  23 

7.4  Evaluation  of  the  ADSDP  Relative  to  Other 

Marine  Corps  Projects  24 

7.4.1  The  Generation  of  Resource  Allocation 

Alternatives  for  Evaluation  24 

7.4.2  Evaluation  of  the  Alternatives 27 

BIBLIOGRAPHY  31 

APPENDIX  A.  QUESTIONS  FOR  USE  IN  REVIEWING  THE  ADEQUACY  OF  THE 

REQUIREMENTS  STATEMENT  (RS)  A-l 

APPENDIX  B.  QUESTIONS  FOR  USE  IN  REVIEWING  THE  ADEQUACY  OF  THE 

FEASIBILITY  STUDY  (FS)  B-l 

APPENDIX  C.  QUESTIONS  FOR  USE  IN  REVIEWING  THE  ADEQUACY  OF  THE 
. ADSDP C-l 

APPENDIX  D.  QUESTIONS  FOR  USE  IN  EVALUATING  THE  CDPA  RESOURCE 

ALLOCATION  ALTERNATIVES  D-l 


vii 


l 


HraCKDlNO  PAIS  BLaMC-NOT  filmed 


LIST  OF  FIGURES 


3-01  ADS  Project  Evaluation  Flow  Chart 


7 


LIST  OF  TABLES 


7-01  Sample  Table  of  Alternatives  To  Be  Considered 25 

7-02  Sample  Display  of  Evaluation  Results  29 


ix 


SECTION  1. 


INTRODUCTION 


Automated  data  systems  (ADS)  have  become  an  important  adjunct  to 
effective,  efficient  mission  accomplishment  in  the  Marine  Corps.  As 
their  great  potential  as  management  tools  has  become  increasingly 
apparent,  the  number  of  proposals  to  develop  new  AD  systems  to  solve  a 
multitude  of  management  problems  has  similarly  grown.  The  realities  of 
military  budgeting,  however,  will  not  permit  the  approval  and  implementa- 
tion of  more  than  a few  of  the  most  promising  and  cost-effective  among 
the  systems  proposed.  Also,  because  some  of  the  ADS  proposals  promise 
very  large  cost  savings  for  the  Corps,  it  is  important  that  a rational 
methodology  be  used  to  evaluate  and  approve  ADS  concepts  to  give  strong 
assurance  that  high  potential  developments  will  not  be  ignored  or  delayed, 
and  that  low  potential  developments  will  not  be  too  readily  accepted  for 
implementation  at  the  expense  of  more  promising  ones. 

The  objective  of  this  volume  is  to  present  a rational  methodology 
to  be  used  for  this  purpose. 


SECTION  2. 


MARINE  CORPS  OBJECTIVES  AND  ADS 


The  development  of  Marine  Corps  ADS  is  clearly  not  an  end  in  itself; 
rather,  it  is  merely  a means  to  attain  some  higher  objectives  related 
to  the  Marine  Corps'  basic  purposes.  It  is  important  that  those  higher 
objectives  be  kept  constantly  in  mind  when  evaluating  the  potential  worth 
of  a proposed  ADS  development. 

The  Marine  Corps  Mid-Range  Objectives  Plan  (MMROP) , dated  7 October 
1975,  presents  a statement  of  the  "Basic  Marine  Corps  Objectives."  The 
major  theme  is  that  the  fundamental  goal  of  the  Marine  Corps  is  the  main- 
tenance of  a force  that  is  ready,  responsive,  and  willing  to  fight  when- 
ever and  wherever  called  upon.  Well-conceived  ADS  can  help  fulfill  this 
goal  by  improving,  for  example,  the  fiscal,  logistics,  and  personnel 
management  functions  that  support  the  Marine  in  the  field.  Ill-conceived 
ADS  can  hurt  the  combat  Marine  by  imposing  unnecessary  burdens  on  him, 
or  by  consuming  resources  that  might  have  been  used  for  something  that 
would  have  helped  him  more. 

Evaluators  will  often  find  it  very  difficult  to  relate  a proposed 
ADS  or  ADS  feature  to  these  overall  Marine  Corps  objectives  because  of 
their  complicated  relationships,  which  are  often  impossible  to  analyze; 
however,  the  attempt  is  likely  to  produce  a more  beneficial  result  than 
if  those  objectives  were  ignored. 
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SECTION  3. 


DECISION-MAKING  IN  CONCEPT  PHA'"'1  ACTIVITIES 


Section  2.1  of  Volume  I provides  an  overview  of  the  USMC  ADS  develop- 
ment process  and  the  role  of  its  first  component:  the  Concept  Phase. 
During  that  Phase,  the  idea  for  a new  ADS  is  generated  and  may  become  the 
subject  of  a formal  Requirements  Statement  (RS) . After  appropriate 
approvals,  the  Requirements  Statement  is  followed  by  a Feasibility  Study 
(FS) , and  ultimately  by  the  ADS  Development  Plan  (ADSDP) . If  the  ADSDP 
is  approved,  physical  implementation  of  the  proposed  system  begins. 

This  event  concludes  the  Concept  Phase  and  initiates  the  Development 
Phase  of  the  development  process. 

The  decision-making  activities  during  the  Concept  Phase  are  the 
subject  of  the  material  presented  in  this  Volume. 

Figure  3-01  portrays  all  of  the  activities  of  ADS  development  in 
the  Concept  Phase.  These  activities  are  of  three  types: 

a.  Activities  connected  with  action  document  preparation. 

b.  Activities  concerned  with  action  document  review  and  evaluation. 

c.  Action  document  approval/disapproval. 

The  first,  the  document  preparation  procedures,  are  treated  in  detail  in 
Volume  I.  The  primary  emphasis  in  this  volume  is  on  the  review  and 
evaluation  activities  and  the  approval/disapproval  activities. 

Down  the  left  side  of  the  figure  are  shown  the  several  groups  active 
in  the  Concept  Phase  process.  In  the  horizontal  row  to  the  right  of 
each  group  are  the  development  steps  that  group  performs.  These  steps 
are  carried  out  in  the  sequence  indicated  by  the  arrows  connecting  the 
boxes . 

The  requirement  or  need  for  a change  or  improvement  in  information 
processing  may  be  first  recognized  anywhere  within  the  USMC.  Thus,  the 
originator  of  an  RS  can  be  any  person  or  group  anywhere  in  the  Marine 
Corps'  organization,  and  the  initial  critical  review  and  the  approval/ 
disapproval  of  the  RS  are  performed  by  the  originator's  local  command, 
employing  normal  staffing  and  approval  procedures. 
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The  figure  shows  that  the  C4  Division  of  HQMC  performs  a central 
ADS  coordination  function  from  receipt  of  an  RS  throughout  the  Concept 
Phase.  That  is  in  keeping  with  its  assigned  responsibility  to  coordinate 
computing  and  ADS  development  throughout  the  Marine  Corps.  C4  does  not, 
however,  itself  approve  or  disapprove  the  action  documents. 

Functional  Managers  who  might  be  affected  by  a proposed  ADS  are 
called  upon  at  the  appropriate  points  to  review  action  documents  that  are 
pending  approval.  Such  reviews  are  handled  by  the  customary  military 
staffing  rather  than  by  specially  devised  ADS  development  procedures. 

At  each  stage  of  approval/disapproval  the  approval  authority  must 
be  an  office  that: 

a.  Has  the  authority  to  direct  that  the  next  stage  of  ADS  develop- 
ment be  accomplished. 

b.  Controls  the  resources  necessary  to  accomplish  the  next  stage 
of  development. 

c.  Has  cognizance  of  the  subject  matter  of  the  proposed  develop- 
ment (and  all  of  the  functional  areas  significantly  impacted). 

The  outcome  of  an  approval  authority's  action  with  respect  to  a document 
can  take  several  forms : 

a.  Unconditional  approval:  The  document  is  accepted  as  a basis 
for  the  next  development  step. 

b.  Approval  contingent  on  the  fulfillment  of  certain  specified 
conditions:  The  document  is  returned  to  the  preparation  team 
with  the  understanding  that,  if  the  conditions  are  filled  by 
the  team,  the  document  will  then  be  accepted  as  a basis  for 
the  next  development  step. 

c.  Provisional  rejection:  The  action  document  and  further  develop- 
ment are  rejected  in  their  present  form,  but  the  document  may 

be  resubmitted  for  reconsideration  after  appropriate  changes 
by  the  preparation  team. 

d.  Outright  rejection:  The  developers  are  directed  by  the  approval 
authority  to  cease  expending  resources  on  the  concept  develop- 
ment . 

In  all  cases  of  rejection,  the  approval  authority  must  inform  the  devel- 
opers of  its  reasons  for  rejection. 
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The  System  Sponsor  is  a person  charged  with  the  responsibility  of 
overseeing  the  preparation  of  the  Feasibility  Study  and  the  ADS  Develop- 
ment Plan.  Ordinarily  he  is  the  representative  of  a Functional  Manager. 
When  multiple  functional  areas  are  concerned  with  the  development,  the 
System  Sponsor  is  usually  the  representative  of  the  Functional  Manager 
most  affected  by  the  development. 

The  study  teams  need  to  have  different  strengths  and  capabilities 
at  different  points  in  the  development  cycle.  Therefore,  constitution 
of  the  teams  will  vary  from  activity  to  activity.  Throughout  the  Concept 
Phase,  the  study  teams  will  have  strong  user  representation. 

The  following  report  sections  describe  the  review  and  approval 
activities  of  Figure  3-01  in  detail  and  show  how  the  decision-maker's 
responsibilities  interplay  with  those  of  the  users/developers,  who 
typically  represent  the  advocates  of  the  proposed  system. 
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SECTION  4. 


PLANNING  THE  DECISION  PROCESS 


As  a basic  management  principle,  all  ADS  proposals,  small  or  large, 
for  the  maintenance  or  modification  of  an  old  system  or  the  development 
of  a new  one,  should  be  reviewed  and  approved  at  some  organizational 
level  above  that  of  the  proposal's  sponsor.  Otherwise,  a great  prolif- 
eration of  uncoordinated  developments  having  uncertain  usefulness  would 
soon  appear.  The  review  and  approval  process  should  in  all  cases  con- 
cern itself  with  the  essential  considerations  contained  in  the  method- 
ology presented  in  this  report : 

• Identification  of  a real  need  (as  in  the  Requirements  Statement 
specified  herein). 

• Consideration  of  the  feasibility  of  alternative  solutions  (as 
in  the  Feasibility  Study  specified  herein). 

• Evaluation  of  the  overall  cost  effectiveness  of  the  preferred 
alternatives  (as  in  the  ADS  Development  Plan  specified  herein). 

However,  the  proposals  relating  to  minor  projects  should  touch  on  each 
of  those  considerations  in  much  less  detail  and  with  much  less  expendi- 
ture of  effort  and  time  than  those  for  major  projects.  Minor  projects 
should  be  reviewed  and  approved  at  lower  levels  in  the  organization. 

Another  dimension  to  be  considered  is  the  basic  character  of  the 
ADS  proposal.  If  it  is  a proposal  for  making  changes  in  the  name  of 
maintenance  of  an  ongoing  system,  whereby  that  system  will  be  restored 
to  operational  status,  or  permitted  to  run,  or  to  meet  its  original 
design  specifications,  then  the  proposal  carries  a certain  presumption 
of  desirability  growing  out  of  the  ongoing  system's  original  approval. 

If  the  proposal  calls  for  the  modification  of  an  ongoing  system,  whereby 
the  original  system  design  specification  is  not  merely  to  be  fulfilled, 
but  is  to  be  changed  in  a significant  way  and  then  fulfilled,  then  the 
proposal  deserves  closer  scrutiny  since  its  new  design  objectives  have 
never  been  reviewed.  Furthermore,  if  the  proposal  calls  for  a completely 
new  system,  it  demands  the  fullest  consideration  and  evaluation  since 
neither  its  utility,  its  feasibility,  nor  its  cost  effectiveness  have 
ever  been  reviewed. 
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The  resources  spent  on  preparing  and  reviewing  an  ADS  proposal  can 
be  looked  upon  as  a kind  of  insurance  against  ultimate  failure  of  the 
proposed  system.  If  the  proposal  preparation,  review,  and  evaluation 
are  done  thoroughly,  the  proposed  system  will  either  be  approved  because 
it  offers  a high  probability  of  success,  or  be  rejected  because  it  does 
not.  Without  this  preparation,  its  chances  of  success  would  remain 
virtually  unknown;  hence,  there  should  always  be  a proposal  preparation 
and  review  for  even  minor  proposals. 

But  what  level  of  resource  should  be  invested  in  such  a preparation 
and  review  process?  It  is  a matter  of  establishing  a rational  relation- 
ship among  the  factors  of  estimated  system  benefits,  estimated  system 
costs,  and  management's  demand  for  a reasonable  return  on  investment. 

If  we  take  the  project's  potential  return  on  investment  (ROI)  as  a 
measure  of  its  merit,  and  assign  a 20%  ROI  as  the  minimum  measure  of 
"success,"*  then  the  basic  purpose  of  the  review  process  is  to  ensure 
that  the  ROI  will  be  maximized,  and  that  it  will  be  no  less  than  20%  (if 
the  project  is  to  be  approved).  In  mathematical  terms: 

ba 

Maximize  ROI  = - — 77^ — =0.20 

LP  Dl 

where 

Ba  « estimated  annual  net  benefits  of  the  proposed  system 
(annual  benefits  less  annual  operating  costs) 

C = cost  of  proposal  preparation  and  review 

Cpj  = cost  of  system  development  and  implementation  following 
a favorable  review 

thus, 

> c . r 
0.20  p DI 

or  Cp  + CDI  “ 5V 

* 

This  corresponds  to  the  five-year  "payback  period  currently  demanded, 
for  example,  of  proposed  Navy  energy  conservation  projects. 
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which  provides  a useful  rule  for  calculating  the  maximum  resources  to  be 
invested  in  proposal  preparation  and  review: 


0^  plays  dual  roles  in  the  equations;  both  of  those  roles  must  be 
carefully  considered  in  choosing  the  proper  level  for  the  Cp  investment. 
Indirectly,  an  increase  in  Cp  can  provide  resources  to  make  a thorough 
analysis  of  the  feasibility  and  potential  benefits  of  alternative  systems. 
That  may  help  ensure  high  values  for  B and  low  values  for  C (thus 

A L/  L 

increasing  the  estimated  ROI)  , and  it  can  provide  the  desired  high  con- 
fidence in  the  estimates.  However,  every  increment  in  Cp  directly 
increases  the  investment  cost  (Cp  + Cpp)  and  reduces  the  ROI,  which  the 
USMC  is  seeking  to  maximize.  Both  influences  must  be  considered  in 
choosing  the  appropriate  level  for  Cp  investment. 

Obviously,  Cp  should  be  set  as  low  as  possible  while  still  remaining 
high  enough  to  perform  its  confidence-producing  function.  For  example, 
a proposed  ADS  promising  annual  net  benefits  (B^)  of  $100,000  if  success- 
ful, and  costing  $300,000  to  develop  and  implement  (C^)  , would  warrant 
an  expenditure  of  u£  to^  $200,000  (or  about  48  man-months)  for  proposal 
preparation  and  review  £s  necessary  to  establish  the  BA  and  values 
with  a high  degree  of  confidence,  or  at  least  to  provide  confidence  that 
5B  will  exceed  C by  a substantial  margin.  If  the  circumstances  are 

A UI 

such  that  the  attainment  of  that  relationship  appears  doubtful,  then  the 
substantial  expenditure  of  Cp  to  prove  it  or  disprove  it  is  called  for. 

If,  on  the  other  hand,  it  appears  highly  probable  to  everyone  concerned 
that  SB^  will  exceed  CDp,  then  a minimal  effort  on  Cp  is  appropriate, 
and  the  ultimate  ROI  will  be  increased  by  the  saving  of  Cp  resources. 

In  applying  these  rules  to  specific  ADS  proposals,  it  should  be 
noted  that  the  annual  benefits,  B , used  in  the  equation  refer  to  the 

A 

net  annual  benefits  (in  excess  of  annual  operating  costs)  that  accrue 
specifically  to  the  proposed  change.  Thus,  in  the  case  of  a maintenance 
proposal,  B^  would  represent  the  net  incremental  benefit  to  the  USMC  of 
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bringing  the  current  system  up  to  its  original  design  specifications,  not 
the  total  benefits  of  the  system  as  a whole.  Similarly,  in  a modification 
proposal,  would  represent  the  net  incremental  benefits  accruing  to  the 
improvement  of  the  system  beyond  its  original  design  specifications.  In 
the  case  of  a new  system,  naturally  refers  to  the  total  benefits  of 
the  new  system,  because  no  predecessor  system  exists. 

At  what  level  in  the  organization  should  the  authority  for  final 
approval  of  an  ADS  proposal  be  assigned?  Large  projects  that  might  have 
an  important  effect  on  the  Marine  Corps,  either  financially  or  operation- 
ally, should  be  approved  at  higher  echelons  than  small  projects.  It  is 
suggested  that  any  ADS  proposal  (whether  for  the  maintenance  or  modifica- 
tion of  a current  system,  or  for  a new  system)  in  which  the  cost  of 
development  and  initial  implementation  (C^)  is  estimated  at  $100,000  or 
more  should  be  considered  to  be  a major  proposal.  It  should  therefore 
be  processed  fully  in  accordance  with  the  methodology  described  in  these 
volumes,  using  the  previously  described  decision  rule  as  a guide  to  the 
amount  of  preparation  and  review  to  be  invested.  Proposals  whose  is 

in  the  range  $20,000  to  $100,000  should  be  approved  by  the  Headquarters 
Functional  Manager,  whose  DP  resources  would  be  primarily  required  to 
carry  out  the  development  of  the  proposal  if  it  is  approved.  Proposals 
for  Cpj  values  of  less  than  $20,000  should  be  approved  by  the  originator's 
local  command  with  an  after-the-fact  record  report  to  the  affected 
Headquarters  Functional  Manager (s).  In  all  such  proposals,  regardless 
of  size,  the  subjects  specified  in  the  methodology  presented  herein 
should  be  addressed,  the  depth  of  such  analyses  to  be  determined  from 
the  decision  rule. 

Thus,  a project  costing  $100,000  or  more  would  be  processed  fully 
according  to  the  21  steps  prescribed  in  Figure  3-01.  A project  between 
$20,000  and  $100,000  would  follow  the  same  general  procedure  except  that 
the  "functional  manager"  and  the  "approval  authority"  of  Figure  3-01 
would  become  the  same  office,  and  steps  3,  10,  and  16  would  not  be 
applicable.  A project  costing  less  than  $20,000  would  follow  the  same 


i 


14 


general  procedure  as  shown  in  Figure  3-01,  except  that  the  local  com- 
mander in  this  case  would  be  authorized  to  perform  all  the  functions  of 
the  HQMC  offices  shown  in  the  figure.  In  this  situation,  steps  3,  4, 

5,  6,  10,  12,  and  16  would  not  be  applicable. 

Certain  special  situations  would  qualify  as  exceptions  to  the  pre- 
ceding general  rules.  When  an  ADS  is  embedded  as  a subset  in  a larger 
weapon  system  or  is  a specialized  tool  for  a specific  research  and 
development  project,  and  performs  a single  function  that  has  no  signif- 
icant utility  outside  its  role  in  that  particular  context,  it  should  be 
proposed  and  reviewed  as  a part  of  the  normal  procurement  review  pro- 
cedures for  the  larger  system  of  which  it  is  a part , rather  than  as 
specified  in  the  ADS  methodology  presented  herein.  During  that  review, 
however,  the  subjects  addressed  in  the  methodology  of  these  volumes 
should  be  given  consideration,  when  applicable.  Also,  any  systems  whose 
detailed  specifications  are  directed  by  higher  authority  need  not  be 
reviewed  according  to  this  methodology  since,  under  those  circumstances, 
no  decision  options  exist. 


SECTION  5. 


THE  RS  DECISION 


When  the  RS  is  completed  (Box  1 of  Figure  3-01) , it  is  submitted 
to  the  originator (s) ' local  command  for  critical  review,  and  approval  or 
disapproval  (Box  2).  (Appendix  A provides  a list  of  questions  for  use 
by  the  local  command  in  reviewing  the  adequacy  of  the  RS.) 

One  possible  outcome  of  the  review  is  the  rejection  of  the  RS  and 
the  discontinuation  of  any  further  effort  on  the  idea  as  a result  of  the 
reviewer's:  (1)  determination  that  the  user  requirements  cannot  be  suf- 

ficiently defined  or  sufficiently  validated  to  justify  further  develop- 
ment, or  (2)  judgment  that  the  resources  required  for  subsequent  processing 
of  the  idea  are  too  high  to  justify  further  development. 

If  the  RS  is  approved  by  the  local  command,  it  is  forwarded  with  the 
local  command's  comments  through  channels  to  the  Command,  Control, 
Communication,  and  Computers  (C4)  office  at  Marine  Corps  Headquarters 
(Box  3),  which  is  responsible  for  Headquarters  staff  coordination  of  all 
Marine  Corps  ADS  activities.  Headquarters  action  at  this  stage  of  the 
process  is  necessary  to  bring  the  RS  to  the  attention  of  approval 
authorities  who:  (1)  have  a complete  staff  of  functional  experts  capable 
of  discerning  any  Marine-Corps-wide  impacts  that  the  proposed  idea  might 
entail,  and  (2)  have  the  authority  to  provide  resources  in  the  amount  and 
kind  required  to  perform  the  Feasibility  Study,  should  the  RS  be  approved. 

Upon  receipt  of  the  RS,  C4  determines  which  functional  area  (e.g., 
fiscal,  manpower,  logistics,  aviation)  would  be  most  affected  by  the 
proposed  idea,  and  assigns  approval  authority  to  that  office.  C4  then 
routes  the  RS  for  critical  review  to  all  Functional  Managers  at  Head- 
quarters whose  functional  areas  would  be  significantly  impacted  by  the 
proposed  idea  (Box  4).  The  questions  in  Appendix  A are  for  the  reviewers' 
use.  The  purpose  of  the  review  is  to  critique  the  RS  and  either  support 
or  oppose  it  as  a basis  for  further  development.  The  reviewers  will: 

• Pass  judgment  on  the  validity  of  the  user  requirements  in  the  RS . 

• Analyze,  and,  if  necessary,  suggest  adjustments  in  the  estimates 
made  in  the  RS  for  the  resources  required  to  perform  the  Feasi- 
bility Study  and  ADS  Development  Planning. 
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• Determine  whether  the  expenditure  of  resources  to  produce  an  FS 
and  an  ADSDP  is  warranted . 

C4  integrates  the  reviewer's  responses  (Box  5)  and  submits  them 
with  the  RS  to  the  RS  approval  authority  for  decision  (Box  6).  Approval 
at  this  point  does  not  authorize  development  of  an  ADS  but  only  commits 
resources  for  the  next  development  subphase:  the  Feasibility  Study. 
Rejection  will  terminate,  either  finally  or  provisionally,  any  further 
development.  It  is  the  task  of  the  approval  authority  to: 

1.  Reject  the  RS , or 

2.  Approve  the  RS , commit  resources  for  a Feasibility  Study,  and 
appoint  a System  Sponsor  (Box  7). 

The  System  Sponsor  will  be  responsible,  under  the  auspices  of  the  RS 
approval  authority,  to  organize  the  Feasibility  Study  (FS)  Team  (Box  8) 

[and  lead  the  further  development  effort  as  its  chairman.  He  will  nor- 
mally have  a broad  background  in  Marine  Corps  operations  and  in  ADS 
applications.  If  practicable,  he  should  be  assigned  full-time  to  his 
System  Sponsor  tasks. 


In  all  cases,  regardless  of  the  findings  of  the  Feasibility  Study 
Team,  an  FS  document  will  be  prepared  and  submitted  for  consideration  by 
approval  authorities. 


SECTION  6.  THE  FS  DECISION 
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When  the  FS  is  completed  (Box  9),  it  is  submitted  to  C4  for  assign- 
ment to  an  FS  approval  authority  (Box  10).  The  FS  approval  authority 
will  normally  be  the  same  office  as  the  RS  approval  authority.  In  a rare 
case  it  could  be  a different  office  if  the  perceived  principal  impact  of 
the  proposed  ADS  has  shifted  to  a new  functional  area  as  a result  of  new 
system  features  introduced  by  the  FS.  C4  then  routes  the  FS  for  critical 
review  to  all  Functional  Managers  whose  functional  areas  would  be  signif- 
icantly impacted  by  the  proposed  system  (Box  11).  (The  questions  in 
Appendix  B are  for  the  reviewers'  use  in  this  process.) 

The  reviewers  should  include  functional  area  specialists  attuned  to 
questions  of  operational  feasibility  and  benefits,  and  technical  special- 
ists attuned  to  questions  of  technical  feasibility  and  costs. 

The  reviewers  will: 

a.  Pass  judgment  on  the  adequacy  of  the  analyses,  estimates,  and 
recommendations  contained  in  the  FS. 

b.  If  the  FS  is  acceptable  and  the  selected  solution  approach(es) 
are  ADS  approach(es) , endorse  the  selected  approach(es)  for 
further  development. 

c.  Reconsider,  and,  if  necessary,  recommend  adjustments  in  the 
estimates  of  resources  required  to  carry  out  ADS  Development 
Planning  for  the  endorsed  ADS  solution  approaches. 

C4  integrates  the  reviewers'  responses  (Box  12)  and  submits  them 
with  the  FS  to  the  FS  approval  authority  for  decision  (Box  13) . 

Disapproval  of  the  FS  terminates,  either  finally  or  provisionally, 
further  ADS  development.  Approval  of  an  FS  whose  selected  approach(es) 
are  non-ADS  approach(es)  also  terminates  further  ADS  development. 

Approval  of  an  FS  whose  selected  approach(es)  are  ADS  approach(es)  does 
not  authorize  development  of  the  ADS,  but  only  commits  resources  for  the 
ADS  Development  Planning  subphase.  Upon  approval  of  the  FS,  the  System 
Sponsor  organizes  the  ADSDP  Study  Team  (Box  14) , which  requires  a broader 
range  of  expertise  among  its  members  than  did  the  FS  Team  because  of  the 
technical  demands  of  producing  the  ADSDP  and  its  supporting  documents 
(Box  15).  The  supporting  documents  include  the  system  Functional  Descrip- 
tion (FD),  Data  Requirements  Document  (RD) , ADP  Equipment  Specification 
(ADPES),  and  the  Economic  Analysis  (EA). 


SECTION  7. 


THE  ADSDP  DECISION 


7.1  General.  This  section  describes  the  procedure  to  be  used  in  evalu- 
ating the  ADSDP  and  approving  or  disapproving  its  implementation.  The 
procedure  consists  of  the  following  elements: 

a.  Preliminary  review  of  the  ADSDP  by  the  C4  office  at  HQMC. 

b.  Organization  of  an  ad  hoc  ADSDP  Evaluation  Team  under  C4  chair- 
manship at  HQMC. 

c.  The  Team's  evaluation  of  the  ADSDP  on  its  own  merits  as  a cost- 
effective  proposal. 

d.  The  Team's  development  of  a range  of  alternative  allocations 
(of  the  manpower  resources  at  the  affected  CDPA)  to  the  ADS 
projects  competing  for  support. 

e.  The  Team's  evaluation  of  the  relative  merits  of  those  alterna- 
tives. 

f.  The  approval  authority's  decision  regarding  further  implementa- 
tion of  the  ADSDP. 


7.2  Formation  of  the  ADSDP  Evaluation  Team.  When  a new  ADS  Development 
Plan  is  received  in  C4  that  office  will  conduct  a preliminary  review  of 
the  ADSDP  to  ensure  that  it  has  been  prepared  completely  and  accurately 
in  accordance  with  procedural  instructions  for  ADSDP  preparation.  If  the 
ADSDP  fails  in  the  review,  it  will  be  returned  to  the  development  team 
for  correction  and  resubmission.  If  it  passes  the  preliminary  review, 
the  Director  of  C4  will  designate  an  ADSDP  approval  authority  (Box  16)  , 


and  will  send  a copy  of  the  ADSDP  to  each  Headquarters  Functional  Manager 


with  the  request  that  he  review  it  for  any  possible  impact  on  his  func- 
tional area.  If  none  of  the  nonsponsoring  Functional  Managers  sees  any 
impact  on  his  functional  area,  the  sponsoring  Functional  Manager  will 
unilaterally  conduct  the  ADSDP  evaluation.  If  a nonsponsoring  Functional 
Manager  finds  that  the  proposed  development  will  have  a significant  impact 
on  his  functional  area,  he  will  assign  a representative  from  his  functional 
area  to  act  as  his  representative  on  an  ad  hoc  ADS  Evaluation  Team,  which 
will  be  convened  under  the  chairmanship  of  a C4  representative  (Box  18) 
to  evaluate  the  ADSP  (Box  19). 


It  is  important  that  the  members  of  the  Evaluation  Team  be  carefully 
selected  to  perform  this  important  role.  Functional  Managers  who  are 
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affected  should  select  as  their  representatives  people  who  possess  the 
following  qualifications: 

• An  appreciation  of  FMF  operational  problems  (to  provide  an  under- 
standing of  the  advantages  and  disadvantages  that  AD  systems  may 
offer  to  Marine  combat  units). 

• An  expert  knowledge  of  the  major  mission  programs  and  the  ADS 
supporting  programs  sponsored  by  the  functional  areas  they  repre- 
sent. 

• Good  oral  and  written  communications  skills. 

• Of  the  same  or  nearly  the  same  rank  as  other  team  members  (to 
avoid  any  rank-generated  bias  in  the  evaluation  results) . 

• Not  members  of  the  prospective  ADS  user  group,  of  the  Sponsor's 
immediate  office,  or  of  any  of  the  teams  that  have  previously 
prepared  inputs  to  the  ADSDP  under  consideration,  or  have  pre- 
viously reviewed  these  inputs  at  the  preceding  Requirements 
Statement  or  Feasibility  Study  approval  points.  (The  purpose  of 
this  provision  is  to  ensure  as  completely  as  possible  the  absence 
of  bias  growing  out  of  former  personal  identification  with  the 
ADSDP  under  consideration.) 

The  Chairman  of  the  Evaluation  Team  will  assign  as  members  one  or 
more  technical  specialists  in  the  areas  of  computers,  data  communications, 
and  economic  analysis,  including  at  least  one  representative  of  the 
Central  Design  and  Programming  Activity  (CDPA)  at  which  the  proposed  sys- 
tem would  be  developed,  if  approved. 

It  should  be  recognized  that  portions  of  the  ADSDP  under  considera- 
tion by  the  Evaluation  Team  will  have  already  received  substantial  review 
and  approval  at  the  preceding  Requirements  Statement  and  Feasibility 
Study  approval  points,  as  well  as  the  expert  attentions  of  the  Development 
Team  during  preparation  of  th°  ADSDP  and  its  supporting  documents. 

The  proposed  ADS  can.  therefore,  be  assumed  to  be  a system  whose 
usefulness  and  feasibility  as  an  individual  system,  have  already  been 
well  demonstrated;  otherwise  it  would  not  have  cleared  the  earlier  reviews 
and  been  approved  for  the  substantial  expenditure  of  resources  that  were 
required  to  prepare  the  ADSDP.  The  main  purpose  of  the  ADSDP  Evaluation, 
then,  is  to  determine  (1)  whether  the  proposed  ADS  is  worthwhile  in  terms 
of  the  benefits  it  promises  for  the  resources  it  demands  and  (2)  how  it 
fits  into  the  larger  Marine  Corps  picture  in  terms  of  its  competition  for 
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the  limited,  virtually  fixed,  manpower  resources  allocable  to  ADS 
development  at  the  sponsoring  Functional  Manager's  CDPA. 


7.3  Analysis  of  the  ADSDP  on  its  Own  Merits.  The  first  step  of  the 
Evaluation  Team's  review  of  the  ADSDP  is  devoted  to  an  analysis  of  the 
worth  of  the  proposed  system  on  its  own  merits,  not  yet  considering  its 
competition  (for  resources)  with  other  Marine  Corps  projects.  (This  com- 
petition is  treated  under  the  second  step  of  the  evaluation,  and  is  dis- 
cussed in  the  following  section  of  this  report.) 

For  this  purpose,  each  member  of  the  Evaluation  Team  will  be  requested 
by  the  Chairman  to  analyze  carefully  the  incoming  ADSDP,  addressing  at 
least  the  questions  listed  in  Appendix  C of  this  report.  The  Chairman  will 
then  convene  a meeting  of  the  Team,  wherein  each  member  will  present  orally 
his  considered  position  as  to  whether  the  ADSDP:  (1)  adequately  presents 
a valid  case  for  an  AD  system,  that  (2)  appears  to  be  a worthy  (i.e.,  cost- 
effective)  candidate  for  further  consideration  in  the  second  phase  of  the 
evaluation.  After  the  general  discussion,  a consensus  among  the  members 
will  determine  the  Team's  answer  to  the  above  two  questions.  If  the  Team 
decides  that  the  ADSDP  is  lacking  in  either  respect , the  ADSDP  will  be 
returned,  with  documented  reasons  for  its  rejection,  to  the  System  Sponsor 
for  reconsideration  and  possible  resubmission  at  the  Sponsor's  option. 

On  the  other  hand,  for  compelling  reasons  that  it  will  document,  the  team 
may  simply  reject  the  ADSDP  as  being  unworthy  of  further  support,  and  will 
return  it  to  the  System  Sponsor  via  the  ADSDP  approval  authority. 

Rejection  of  the  ADSDP  would  be  unlikely  under  static  conditions. 
However,  given  the  real-world  conditions  of  changing  internal  requirements, 
as  well  as  external  directives,  the  underlying  assumptions  or  conditions 
for  which  the  earlier  FS  and  RS  approvals  were  given  could  now  be  so  sig- 
nificantly changed  as  to  effectively  void  the  earlier  approvals. 

If  the  Team  decides  that  the  ADSDP  is  acceptable  in  both  respects, 
it  will  be  further  processed  under  the  second  step  of  the  evaluation,  as 
described  below. 
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7.4  Evaluation  of  the  ADSDP  Relative  to  Other  Marine  Corps  Projects. 

The  second  step  of  the  evaluation  compares  the  ADSDP  with  other  ongoing 
projects  and  considers  several  alternative  allocations  among  them  of  the 
available  ADS  resources,  principally  manpower  resources. 

7.4.1  The  Generation  of  Resource  Allocation  Alternatives  for  Evaluation. 

At  a typical  CDPA,  the  available  manpower  resources  at  any  given  time 
might  be  allocated  to  one  large  development  project,  a few  medium-sized 
projects,  and  a large  number  of  small  projects  devoted  to  maintenance 
and/or  modification  of  existing  ADS,  each  bearing  its  own  priority  rela- 
tive to  the  total  project  population. 

Because  the  proposed  ADS  is  in  competition  (with  other  systems  under 
development  at  the  sponsor's  CDPA)  for  available  development  resources,  in 
the  second  step  of  the  evaluation  the  alternatives  at  issue  make  up  the 
several  ways  that  the  proposed  system  development  might  be  introduced  into 
the  CDPA.  The  possibilities  range  from  full-scale,  full-speed  development 
down  to  postponed  development  (i.e.  , where  the  proposed  system  must  wait 
in  queue  until  the  required  resources  can  be  made  available  for  its  develop- 
ment). 

The  Team's  next  order  of  business  is  to  establish  a systematic  range 
of  alternative  ways  to  introduce  the  proposed  new  project  (or  delay  its 
introduction).  The  alternatives  are  designed  by  the  Team  to  indicate,  in 
effect,  a range  of  priorities  that  might  be  assigned  to  the  proposed  new 
project,  in  relation  to  its  competitors,  based  on  an  overall  evaluation 
of  the  merit  of  each  of  the  competing  projects.  Thus,  if  the  proposed 
project  is  considered  more  important  (i.e.,  more  meritorious)  than  its 
competitors,  it  will  be  assigned  a more  generous  allocation  of  the  avail- 
able system  development  resources  (at  the  expense  of  its  competitors)  and 
vice-versa. 

Table  7-01  is  a sample  of  the  outcome  of  this  process,  as  it  would 
appear  if  the  new  project  were,  for  example,  sponsored  by  the  DCS  for  I&L, 
and  were  being  evaluated  by  the  Evaluation  Team  for  possible  development 
at  the  I&L  CDPA  activity  at  Albany,  Georgia.  In  the  left  column  are  listed 


24 


the  projects  currently  under  development  at  the  CDPA,  as  well  as  the 
(hypothetically)  proposed  ADS  "MCPEM. "* 

a.  Alternative  1 — Current  Status.  Under  Alternative  1,  the  current 
allocation,  are  entered  the  levels  of  effort  currently  devoted 
to  these  projects,  presented  as  a percentage  of  total  man-hours 
available  for  development  work  at  the  CDPA.  Under  this  Alterna- 
tive, the  effort  devoted  to  Project  "MCPEM"  is  zero  because  it 
has  not  yet  been  approved  for  implementation. 

b.  Alternative  2 — High  Priority  to  New  Project.  Under  Alternative 
2,  the  Evaluation  Team  gives  highest  priority  to  "MCPEM"  and 
allocates  the  full  amount  of  manpower  resources  that  would  be 
necessary  to  develop  "MCPEM"  on  an  optimal  schedule  to  ensure 

its  earliest  possible  operational  capability  (30%  in  our  example) . 
The  resource  allocations  to  the  current  projects  must,  of 
course,  be  scaled  down  appropriately  to  make  these  resources 
available  for  Project  "MCPEM."  The  necessary  scaling  down  can 
usually  be  accomplished  readily  by  generally  following  the 
relative  priorities  that  the  current  projects  have  earlier  been 
assigned. 

c.  Alternative  3 — Medium  Priority  to  New  Project.  Alternative  3 
is  designed  to  give  support  to  "MCPEM"  at  an  intermediate  level 
between  those  assigned  to  "MCPEM"  under  Alternatives  2 and  4. 

This  alternative  would  provide  a development  pace  for  "MCPEM" 
that  is  inferior  to  optimal  Alternative  2 in  terms  of  efficiency 
and  completion  date,  but  is  superior  to  the  minimally  acceptable 
program  represented  by  Alternative  4. 

d.  Alternative  4 — Low  Priority  to  New  Project.  Under  Alternative 
4,  "MCPEM"  is  allocated  the  minimum  resources  considered  neces- 
sary to  make  it  a successful  project;  i.e.  , if  fewer  resources 
were  allocated,  the  development  process  would  be  unacceptably 
inefficient,  or  the  promised  operational  capability  would  be  made 
available  unacceptably  late,  perhaps  so  late  as  to  render  its 
product  useless  to  the  Marine  Corps. 

e . Alternatives  5 and  6 — Additional  Manpower  for  New  Project^. 

As  noted  in  Table  7-01,  Alternatives  1 through  4 are  designed 
on  the  assumption  that  only  the  current  CDPA  manpower  resources 
are  available  to  accommodate  "MCPEM"  along  with  the  other 
projects  currently  underway.  Two  additional  alternatives  must, 
therefore,  be  added  to  represent  the  possibility  that  the  cur- 
rent projects  might  be  maintained  at  or  near  their  currently 


*"MCPEM"  is  only  an  illustrative  acroynm. 

^These  alternatives  snould  be  used  if  additional  resources  can  be  obtained 
for  either  the  new  project  or  other  projects  competing  for  the  same 
manpower. 


assigned  levels  of  effort  without  interference,  while  the  new 
project  "MCPEM"  might  be  developed  by  adding  new  personnel  to 
the  CDPA  through  personnel  transfer,  for  example,  (as  repre- 
sented by  Alternative  5)  or  by  use  of  external  contractors  (as 
represented  by  Alternative  6). 

Before  the  ADSDP  evaluation  task  can  begin,  the  Evaluation  Team 
must  arrive  at  a consensus  on  the  definitions  of  the  alternatives  to  be 
evaluated.  With  the  above  general  guidelines  in  mind,  the  Chairman  of 
the  Evaluation  Team  will  issue  to  each  team  member  a worksheet  in  the 
form  of  Table  7-01,  having  filled  in  only  the  identifying  blanks  at  upper 
left  and  the  numerical  entries  under  Alternative  1 (which  reflect  the 
current  factual  status  of  workload  at  the  affected  CDPA).  Each  team 
member  will  then  be  requested  to  complete  the  numerical  entries  under 
Alternatives  2 through  6,  using  his  best  judgment  to  reach  an  appropriate 
balance  among  the  competing  systems.  When  these  have  been  returned  to 
the  Chairman,  they  will  be  compared  in  a general  discussion  by  the  whole 
team.  Each  member  will  be  encouraged  to  present  his  reasons  for  reaching 
his  choices  on  the  alternatives.  The  discussions  will  continue  until  a 
clear  consensus  is  reached  wherein  a single  set  of  numerical  entries 
(perhaps  even  a set  which  is  different  from  any  of  those  submitted 
originally)  is  acceptable  to  the  majority  of  team  members.  If  no  clear 
consensus  emerges,  the  Chairman  will  choose  the  final  set  of  numerical 
entries. 

If  possible,  more  than  six  alternatives  for  evaluation  should  be 
avoided  because  cf  the  increased  complexity  of  dealing  with  a large  num- 
ber of  alternatives  in  the  evaluation. 

7.4.2  Evaluation  of  the  Alternatives.  Once  the  alternatives  have  been 
defined,  the  team  members  must  determine  their  relative  values  to  the 
Marine  Corps  so  that  the  most  valuable  alternative  may  be  chosen  for 
implementation.  The  Chairman  will  ask  each  team  member  to  rate  the  six 
alternatives  subjectively  on  a scale  of  0 to  100  (assigning  the  grade 
100  to  the  best  alternative),  after  asking  himself  the  questions  listed 
in  Appendix  D. 
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Each  team  member  will  submit  his  ratings  of  the  six  alternatives, 
together  with  a written  presentation  of  his  analysis  of  the  advantages 
and  disadvantages  of  each  alternative,  to  the  Chairman.  Copies  of  all 
submissions  will  be  provided  to  each  team  member,  and  a general  meeting 
to  discuss  them  will  be  convened  by  the  Chairman  with  a view  toward 
reaching  a consensus.  At  the  meeting,  the  proponent  of  any  exceptional 
point  of  view  will  orally  present  his  rationale  and  be  subjected  to 
questions  from  his  fellow  team  members.  If  Alternatives  5 or  6 are 
advocated,  the  proposed  sources  of  additional  manpower  or  of  contract 
funds  must  be  identified.  When  there  are  no  more  questions,  another 
iteration  of  the  rating  process  will  be  performed,  in  the  meeting  room  if 
desired,  and  the  Chairman  will  compile  the  results,  including  all  the 
numerical  ratings  assigned  by  each  team  member.  To  reach  the  final  choice, 
the  Chairman  will  average  the  numerical  ratings  of  all  members  for  each 
alternative.  The  final  ranking  of  alternatives  will  be  based  upon  the 
average  ratings,  and  the  alternative  with  the  highest  average  numerical 
rating  will  be  the  Team's  final  choice.  The  numerical  ratings  provided 
by  each  team  member  will  be  preserved  to  indicate  the  range  of  uncertainty 
that  existed  in  the  Team's  collective  judgment. 

Table  7-02  is  a sample  display  of  evaluation  results  in  which  Alter- 
native 5 would  be  chosen. 

The  results  will  be  documented  by  the  Chairman  and  submitted  to  the 
ADSDP  approval  authority  for  decision  (Box  20). 

If  the  ADSDP  is  approved,  C4  will  coordinate  the  necessary  imple- 
menting action,  utilizing  resources  authorized  by  the  approval  authority 
(Box  21). 

This  final  step  in  the  ADSDP  approval  process  not  only  evaluates  the 
proposed  ADS  on  its  own  merits  but  also  in  relation  to  its  impacts, 
positive  or  negative,  on  other  ongoing  ADS  development  or  maintenance/ 
modification  projects. 
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Table  7-02 

* 

Sample  Display  of  Evaluation  Results 


Alternatives 


Team  Members 

1 

2 

3 

4 

5 

6 

A 

15 

65 

20 

55 

100 

15 

B 

25 

85 

35 

60 

100 

40 

C 

0 

100 

60 

10 

75 

35 

D 

30 

40 

100 

40 

60 

50 

E 

65 

50 

70 

100 

80 

20 

F 

20 

100 

45 

50 

45 

25 

Average  Rating 

25.83 

73.3 

55.0 

52.5 

76.6 

30.8 

Range  of  Ratings 

0-65 

40-100 

20-100 

10-100 

45-100 

15-50 

^Entries  indicate  the  team  members'  subjective  evaluations  of  the 
alternatives  on  a scale  of  0-100. 
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Appendix  A 

QUESTIONS  FOR  USE  IN  REVIEWING  THE 
ADEQUACY  OF  THE  REQUIREMENTS  STATEMENT  (RS) 


QUESTIONS  FOR  USE  IN  REVIEWING  THE 
ADEQUACY  OF  THE  REQUIREMENTS  STATEMENT  (RS) 


Is  each  need  adequately  justified? 

Were  user  needs  adequately  analyzed,  e.g.  , by  extensive  interviews  and 
by  "decision  table"  analysis?  Are  the  needs  described  fully,  in  terms 
of  subject  matter,  criticality,  relative  priority,  level  of  detail  or 
aggregation,  frequency  of  reports,  full  or  exception  reporting,  the 
decisions  or  actions  they  support? 

In  what  respects  are  the  existing  systems  considered  to  be  inadequate? 

Are  these  hard  deficiencies  whose  resolution  is  essential  to  Marine 
Corps  effectiveness,  or  are  they  merely  bothersome? 

Has  adequate  consideration  been  given  to  upgrading  the  existing  systems 
(to  overcome  their  deficiencies)  through  non-automated  means  such  as 
personnel  changes,  organizational  changes,  procedural  changes? 

Are  all  obvious  needs  expressed? 

Were  existing  systems  adequately  investigated  for  applicability  in  satis- 
fying the  user's  needs? 

Does  the  validation  of  the  user's  requirements  include  specific  documen- 
tary references  to  the  original  source  of  the  requirement,  such  as  an 
OMB  directive? 

Was  due  consideration  given  to  the  timing  and  frequency  of  the  user's 
data  requirements,  i.e.,  is  there  a statement  as  to  when  each  element  of 
data  input  and  output  is  required  and  whether  at  an  hourly,  daily,  weekly, 
monthly,  or  annual  frequency? 

Is  it  made  clear  by  whom  and  at  what  location  each  element  of  output  data 
is  required? 

Were  the  user's  needs  for  output  data  analyzed  from  the  viewpoint  of  all 
the  functions  the  user  must  perform,  e.g.,  forecasting,  planning,  program- 
ming, budgeting,  authorizing,  monitoring,  evaluating,  reporting,  adminis- 
tering, commanding? 

Were  the  user's  needs  analyzed  with  regard  to  all  the  types  of  resources 
the  user  is  concerned  with,  e.g.,  manpower,  money,  energy,  aircraft, 
ground  vehicles,  shore  installations,  equipment,  ammunition,  supplies? 

If  the  user's  needs  were  not  met,  what  operational  disadvantage  would 
the  Marine  Corps  suffer? 

Which  of  the  user's  requirements  could  be  left  unsatisfied  without  sig- 
nificant detriment  to  the  USMC? 

Are  all  the  documentation  requirements  set  forth  in  the  ADSM  fulfilled? 
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ADEQUACY  OF  THE  FEASIBILITY  STUDY  (FS) 
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QUESTIONS  FOR  USE  IN  REVIEWING  THE 
ADEQUACY  OF  THE  FEASIBILITY  STUDY  (FS) 


Will  the  selected  approach  meet  all  of  the  user  requirements  expressed 
in  the  RS? 

Does  the  selected  approach  produce  information  that  the  intended  user 
does  not  need,  or  will  not  use? 


Does  it  produce  information  that  no  one  needs? 

Is  the  selected  approach  sufficiently  described  to  serve  as  a starting 
point  for  ADS  development? 

Does  the  selected  approach  duplicate  or  overlap  other  ADS  systems  which 
exist  or  are  currently  under  development? 

Does  the  selected  approach  provide  for  adequate  access  (e.g.,  for  input, 
interrogation  of  the  data  base,  output)  by  all  appropriate  potential 
users  in  the  Marine  Corps? 

What  is  the  estimated  life  expectancy  of  the  proposed  system? 

Does  the  selected  approach  require  a high  level  of  expertise  in  pro- 
grammers or  operators? 

What  elements  of  the  existing  system(s)  can  be  carried  over  and  used  by 
the  selected  solution  approach,  e.g.,  talent,  equipment,  forms? 

Has  the  rationale  for  automatic  data  input  from  sensors  been  considered 
so  as  to  economize  on  the  need  for  manual  intervention? 


Will  the  selected  approach's  outputs  be  accurate  enough  to  be  credible 
and  reliable,  and  thus  forestall  the  maintenance  of  redundant,  time- 
consuming  manual  systems  "kept  in  a drawer"? 

Is  the  selected  approach  likely  to  fulfill  the  needs  of  later  incumbents 
in  the  user  organization,  as  well  as  the  current  user/sponsor? 

Is  the  selected  approach  adaptable  to  possible  future  changes  in  planning 
procedures,  or  operations  of  the  user  organization(s) ? 

Will  the  system  be  readily  adaptable  to  wartime  conditions  should  they 
occur? 

How  complex  does  the  selected  approach  appear? 

Are  the  system  techniques  called  for  in  the  selected  approach  all  within 
the  current  state  of  the  art? 
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What  is  the  estimated  probability  of  successful  development  and  imple- 
mentation of  the  selected  approach?  Are  there  substantial  technical  or 
operational  risks  involved? 

Has  the  proposed  system  been  used  in  other  organizations?  With  what 
results? 

Are  the  numbers  and  skills  of  personnel  required  for  development  and 
operation  of  the  selected  approach  currently  available  in  the  user  and 
data  systems  organizations? 

Is  the  selected  approach  inherently  capable  of  handling  the  projected 
workload? 

What  undesirable  impact  may  the  selected  approach  have  on  morale  or 
organizational  effectiveness? 

Has  the  sponsor  of  the  proposed  system  considered  all  the  alternatives 
to  the  proposed  system,  and  evaluated  their  relative  cost-effectiveness? 

Are  all  expected  obstacles  identified  and  evaluated? 

Does  the  selected  approach  impose  a requirement  for  additional  data 
collecting? 

Are  all  the  documentation  requirements  set  forth  in  the  ADSM  fulfilled? 

Will  the  outputs  of  the  selected  approach  be  consistent  with  the  outputs 
of  existing  systems  treating  similar  subject  matter?  If  not,  how  is  it 
proposed  to  resolve  the  inconsistencies? 

Will  the  selected  approach  employ  programming  language  that  is  operable 
in  view  of  the  training  and  background  of  the  USMC  programmers  and 
operators  who  will  use  the  system? 

Is  the  selected  approach  clearly  superior  to  the  other  alternatives? 
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QUESTIONS  FOR  USE  IN 
REVIEWING  THE  ADEQUACY  OF  THE  ADSDP 


Does  the  originally  stated  need  for  the  proposed  system  still  exist? 

How  long  is  it  likely  to  exist? 

Are  any  of  the  user  requirements  in  the  RS  that  cannot  be  satisfied  by 
the  system? 

Can  all  stated  ADS  objectives  be  tested  or  otherwise  checked  for  fulfill- 
ment during  system  development? 

Are  there  adequate  provisions  in  the  proposed  system  for  security: 
physical?  communications?  user  access?  software? 

Is  system  flow  adequately  described  to  ensure  harmony  among  the  various 
processing  steps,  intermediate  user  actions,  sources  and  destinations  of 
inputs  and  outputs,  and  responsibilities  of  organizational  units? 

Are  the  non-automated  elements  of  the  system  integrated  with  the  auto- 
mated elements  to  show  a complete  operating  entity? 

How  survivable  is  the  selected  approach  in  the  event  of  military  action? 

Is  the  system  adequately  protected  from  the  hazards  of  pollution,  rough 
handling,  and  other  environmental  effects? 

Is  the  system  designed  for  easy  maintenance? 

Is  it  reliable? 

Is  there  provision  for  graceful  fallback  of  operations  in  the  event  of 
trouble  in  the  system?  Are  redundancies  or  backups  provided  for  critical 
elements? 

Is  the  system  organizationally  located  for  maximum  benefit  to  the  Marine 
Corps? 

What  provisions  are  made  for  obtaining  the  new  personnel  skills  required 
by  the  system?  for  placement  of  displaced  employees?  for  needed  organiza- 
tional changes? 

Is  the  required  processing  of  each  element  of  input  described? 

Does  the  proposed  system  provide  a level  of  information  quality  that 
exceeds  the  user's  needs? 

Are  system  components  readily  available  on  the  commercial  market  to  meet 
the  planned  development  schedule? 
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Is  the  system  development  schedule  realistic? 

Does  the  proposed  development/ implementation  schedule  meet  the  user's 
requirements  for  initial  availability  of  the  new  system? 

Does  the  proposed  system  provide  a frequency  and  speed  of  response  that 
exceeds  the  realistic  needs  of  the  users? 

What  are  the  expected  benefits  of  the  proposed  system  (a)  to  the  intended 
users?  and  (b)  to  the  Marine  Corps  as  a whole? — in  dollar  savings,  morale, 
inventory  reductions,  personnel  released,  working  conditions,  decision- 
making, response  time,  transportability,  maintainability,  training  time, 
vulnerability,  simplicity,  efficiency,  and  combat  effectiveness  of  the 
FMF.  (The  new  system  may  be  faster  than  the  old,  but  how  important  is 
speed  in  the  intended  application?) 

Other  possible  benefits  might  include: 

Equipment  displacement:  the  value  of  equipment  that  will  be  freed  because 
of  the  proposed  system,  e.g.,  calculators,  accounting  machines,  or 
earlier  generation  computer  systems. 

Direct  cost  displacement:  cost  of  tasks  that  are  currently  being  per- 
formed by  people  and  which  will  be  performed  wholly  or  partially  by  the 
proposed  system,  e.g.,  preparation  of  routine  documents,  collection  and 
summarization  of  data,  analytical  computation. 

Indirect  cost  displacement:  cost  savings  derived  from  the  proposed 
system's  enhanced  information  handling  capabilities  of  speed,  accuracy, 
and  information  retrieval,  e.g.,  reduction  in  inventory  costs. 

Improved  responsiveness  of  the  user  organization  to  the  needs  of  sup- 
ported organizational  units:  due  to  improved  information  regarding  USMC 
requirements,  trends,  problem  areas,  organizational  performance,  etc. 

Improved  managerial  planning  and  control:  quicker  preparation  of  plan- 
ning documents  through  quicker  data  collection  or  through  the  use  of 
planning  models  or  simulations  designed  to  give  answers  to  "what  if?" 
planning  questions  regarding  various  plans,  assumptions  and  contingencies; 
also  more  frequent  iteration  of  planning  cycles;  deeper  insight  into  the 
factors  affecting  organizational  performance. 

Organizational  benefit:  organizational  improvements  resulting  from  the 
new  system,  e.g.,  better  operational  discipline  in  performing  accurately 
and  on-time,  more  flexibility  in  personnel  recruitment  and  assignment. 

Have  these  benefits  been  quantified  in  terms  of  their  dollar  impacts? 

Over  what  time  period  is  the  system  expected  to  pay  back  (through 
savings)  the  required  initial  investment? 
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Which  costs  or  wasteful  practices  associated  with  existing  systems  will 
be  reduced,  eliminated,  or  avoided  by  the  new  system? 

Are  all  costs  of  the  new  system  identified,  e.g.  , for  R&D,  studies, 
system  design,  training,  programmers,  operators,  programing  and  debugging, 
development  of  new  forms,  parallel  operations  of  old  and  new  systems,  site 
preparation,  hardware  and  software  procurement,  hardware  and  software 
maintenance? 

Are  the  estimated  resource  requirements  reasonable  for  the  contemplated 
development? 

What  valuable  resources  other  than  dollars  are  affected  by  the  proposed 
system:  manpower?  energy? 

What  was  the  cost-effectiveness  of  the  existing  system(s)  in  comparison 
with  the  proposed  system? 

Is  the  estimated  system  accuracy  consistent  with  the  purpose  to  which 
the  results  will  be  applied? 

Does  the  system  include  adequate  error  alarms  and  diagnostic  routines 
to  signal  and  trace  any  errors  that  may  occur? 

Is  the  proposed  system  capable  of  performing  useful  tasks  other  than 
those  required  by  the  current  user/ sponsor ? 

Is  there  adequate  provision  for  the  system's  workload  capacity  or  func- 
tional capabilities  to  be  expanded,  such  as  by  adding  modular  subsystems? 

Is  the  system  designed  to  grow  to  meet  the  estimated  future,  as  well  as 
the  current  needs  of  the  intended  user? 

Does  the  system  contribute  to  the  desirable  standardization  of  data 
elements  and  codes,  terminology,  programming  languages,  source  documents, 
systems  documentation,  education  and  training? 

What  safeguards  are  provided  against  failure  of  the  development  project, 
e.g.,  generation  of  test  data,  documentation,  system  modeling,  progress 
reporting? 

What  confidence  level  (in  percent  terms)  is  associated  with  the  plan's 
estimates  of  the  performance  and  cost  of  each  system  element? 

Does  the  plan  identify  goals,  resources,  assumptions,  dates,  responsi- 
bilities, and  measures  of  progress? 

Have  the  required  outputs  been  identified?  What  do  they  contain?  At 
what  time  intervals  are  they  produced?  Where  do  they  go?  What  form  are 
they  in?  Who  is  responsible  for  receiving  them? 
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Have  the  required  data  and  parameter  Input 3 been  identified?  What  do 
they  contain?  At  what  time  intervals  are  they  produced?  Where  do  they 
go?  What  form  are  they  in?  Who  is  responsible  for  generating  them? 

Is  the  proposed  system  compatible  with  other  AD  systems  with  which  it 
must  interact,  e.g.  , in  data  formats,  data  sources,  data  base,  input 
parameters,  programming  languages,  output  schedules,  processing  cycles, 
lag  times,  error  rates? 

Is  the  proposed  system  consistent  with  the  objectives  and  constraints 
set  forth  in  the  USMC  ADS  Plan?  Is  it  compatible  with  the  overall  ADS 
architecture  envisioned  for  the  USMC? 

Does  the  proposal  meet  the  requirements  for  OMB  Circular  A-76,  which  is 
designed  to  encourage  Government  movement  toward  greater  use  of  commercial 
ADP  services  in  lieu  of  Government  ownership  and  operation  of  ADP  facili- 
ties? 

Does  the  proposal  comply  with  the  "Brooks  Bill"  (Public  Law  89-306,  80th 
Congress,  H.R.  4845,  30  Oct.  1965),  which  authorized  and  directed  GSA 
to  coordinate  and  provide  for  the  economic  and  efficient  purchase  of  ADP 
equipment  by  Federal  agencies? 

Is  the  proposal  consistent  with  the  provision  of  Draft  DoD  Instruction 
79XX.XX  of  22  Oct.  1976,  "ADP  System  Life  Cycle  Management,"  and  Draft 
DoD  Instruction  79XX.XX  of  29  Oct.  1976,  "Acquisition  by  Contract  of 
Automatic  Data  Processing  Equipment,  Software,  and  Services"? 

What,  if  any,  organizational  changes  would  be  required  to  implement  the 
proposed  system? 

Are  the  interfaces  between  the  proposed  system  and  other  existing  sys- 
tems fully  described  and  evaluated? 

Are  all  the  ADSDP  supporting  documentation  requirements  set  forth  in 
the  ADSM  fulfilled? 

Does  the  ADS  plan  include  effective  plans  for  system  management,  design, 
development,  testing,  parallel  operation,  conversion,  control,  evaluation, 
team  organization,  documentation,  training,  accounting,  file  retention 
time,  quality  controls,  cost  controls? 
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QUESTIONS  FOR  USE  IN  EVALUATING 
THE  CDPA  RESOURCE  ALLOCATION  ALTERNATIVES 


To  what  extent  does  the  alternative: 

• Provide  a good  balance  between  economy  and  effectiveness  in 
supporting  the  Basic  Marine  Corps  Objectives? 

• Comply  with  real-world  constraints  in  the  availability  of  per- 
sonnel skills,  facilities,  equipment,  and  budget,  and  in  directives 
from  higher  authority?  (If  the  preferred  alternative  does  not 
comply  with  the  directives  from  higher  authority,  a reclam a will 

be  necessary  to  gain  the  higher  authority's  approval  of  the 
deviat ion. ) 

• Simplify  and  facilitate  Marine  combat  operations? 

• Avoid  "gold-plated"  features  that  look  good  at  first  glance  but 
will  not  really  help  FMF  combat  effectiveness?  (For  example, 
delivering  a report  in  one  hour  instead  of  three  days  is  not 
worthwhile  unless  the  added  speed  significantly  improves  the 
quality  of  an  important  decision  that  is  based  on  the  report.) 

• Contribute  to  the  Corps-wide  standardization  of  ADS  programs, 
equipment,  and  procedures? 

• Provide  for  a family  of  USMC  information  systems  that  complement 
rather  than  impede  or  duplicate  one  another? 

Can  the  system  be  funded  within  existing  USMC  budgetary  limitations? 

To  what  degree  can  action  be  taken  to  minimize  the  new  system's  undesir- 
able effects? 

What  will  it  cost  to  minimize  the  new  system's  impact  on  other,  existing, 
systems  or  other  systems  under  development? 

Is  it  amenable  to  economy  through  consolidation  with  other  existing  or 
planned  systems? 

When  and  at  what  level  of  funding  should  the  proposed  system  be  initiated 
so  as  to  minimally  impede  the  existing  AD  systems  or  those  currently 
under  development? 

What  is  the  relative  importance  of  the  output  of  the  proposed  system  com- 
pared with  that  of  other  existing  or  proposed  systems  for  the  same 
organizational  unit? 

What  priority  should  be  assigned  to  the  proposed  system  among  all  proposed 
systems  currently  competing  for  funding  authorization? 
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